Apparatus for three-dimensional measurements of physical characteristics within a data center

ABSTRACT

An apparatus and method for measuring the physical quantities of a data center during operation and method for servicing large-scale computing systems is disclosed. The apparatus includes a cart that supports a plurality of sensors. The cart is moveable within the data center. The sensors capture temperature or other physical parameters within the room. The sensor readings, along with position and orientation information pertaining to the cart are transmitted to a computer system where the data is analyzed to select the optimum temperature or other system environmental parameters for the data center.

This application is a continuation of U.S. patent application Ser. No.11/195,426, filed on Aug. 2, 2005, now U.S. Pat. No. 7,366,632, theentire disclosure of which being herein incorporated by reference in itsentirety. This application is also related to U.S. patent applicationSer. No. 12/110,732, filed on Apr. 28, 2008.

FIELD OF THE INVENTION

This invention generally relates to the field of thermal measurement andmore specifically to thermal measurement of data centers.

DESCRIPTION OF RELATED ART

There are many cases where it is desirable to accurately measure,analyze, and optimize the environmental characteristics of an area. Onesuch area is a data center. A data center is a room wherein rows ofequipment racks and enclosures situated side by side in very largenumbers are located. The equipment racks and enclosures contain andorganize communications and information technology equipment, such asservers, internetworking equipment and storage devices. Each piece ofthe rack-mounted equipment consumes electrical power and generates heat.The amount of heat generated corresponds to the amount of power consumedby each piece of equipment. Naturally, the total heat output of a singlerack is the result of a cumulative affect of the heat generated by eachpiece of rack-mounted equipment. As a result, the heat output of eachrack may vary greatly, depending upon the type of equipment, the dutycycle of use of each piece, the ambient temperature, and especially thecooling system being used.

Heat produced by rack-mounted equipment can have adverse effects on theperformance, reliability and useful life of the equipment components. Inparticular, rack-mounted equipment housed within an enclosure isparticularly vulnerable to heat build-up and hot spots produced withinthe confines of the enclosure during operation.

The problem is compounded by a dramatic surge of power consumption incomputing systems that has significantly increased the costs of cooling,infrastructure, and energy of data centers and supercomputers. Forexample, just 25 years ago the typical dissipated power in a computerrack was only ˜1 kW while today we are reaching power levels of almost40 kW in a similar size rack. It is inevitable that future rack powerlevels will increase even further.

Therefore, the thermal design of these large scale computing systems hasemerged as one of the key challenges for any data center. To this end, amuch more detailed understanding of the thermal implications of thephysical layout is warranted.

Attempts to thermally profile data centers using field data orperforming simulations using computer based models have yieldedunsatisfactory results, lacking the accuracy, turn around, and ease ofinterpretation which is needed to optimize data center layouts.

There is no suitable technique and method available which can readilymap the temperature distribution in three dimensions in the data center.

Therefore a need exists to overcome the problems with the prior art asdiscussed above.

SUMMARY OF THE INVENTION

Briefly, in accordance with the present invention, disclosed is anapparatus and method for measuring physical characteristics, e.g., thethermal distributions and other measurements, such as relative humidity,absolute humidity, barometric pressure, and wind flow rate, wind speedand wind direction, in a data center. In an embodiment of the presentinvention, the system includes a framework with a plurality of sensors.Each sensor is physically coupled to the framework. Each sensor being ata different location on the framework. Each sensor of the plurality ofsensors measures at least one physical characteristic of an environmentwithin a data center.

The system further includes a means for communicating the measuredphysical characteristic from at least one of the plurality of sensors,location information, and/or orientation information to a data storagedevice. The location information can relate to the location of theframework in the data center. It can also relate to the location of atleast one of the plurality of sensors. The orientation information canrelate to the orientation of the framework in the data center. It canalso relate to the orientation of at least one of the plurality ofsensors.

The framework, according to an embodiment, is shaped and dimensioned sothat at least one of the sensors is able to measure a physicalcharacteristic directly above a rack in the data center and on a side ofa rack in the data center.

The framework is provided with a means for movement, and moreparticularly at least one friction reducing device, such as a set ofwheels that allow the framework to be moved to locations within the datacenter. In one embodiment, a motor is couple to and drives the wheels,allowing the apparatus to be positioned within the data center.

In an embodiment of the present invention, a method for measuring atleast one physical characteristic, e.g., the thermal distributionswithin a data center, is disclosed. The method includes placing aphysical/environmental parameter measuring cart within a data center,measuring at least one physical/environmental characteristic within thedata center with at least one sensor on the framework, storing themeasurements of the at least one physical characteristic on adata-storage device, and storing location information, such as locationof the cart within the data center, on the data-storage device.

The method further includes transmitting the measurement data and thelocation information to a remote receiving device.

The foregoing and other features and advantages of the present inventionwill be apparent from the following more particular description of thepreferred embodiments of the invention, as illustrated in theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The subject matter, which is regarded as the invention, is particularlypointed out and distinctly claimed in the claims at the conclusion ofthe specification. The foregoing and other features and also theadvantages of the invention will be apparent from the following detaileddescription taken in conjunction with the accompanying drawings.

FIG. 1 is a perspective view of a typical data center.

FIG. 2 is a perspective view depicting one embodiment of the presentinvention.

FIG. 3 is a cross sectional diagram depicting one embodiment of thepresent invention.

FIGS. 4-7 are illustrations of one embodiment of the present inventionused to measure a set of racks within a data center.

FIG. 8 is a block diagram of a computer system in which an embodiment ofthe present invention can be implemented.

FIG. 9 is a block diagram depicting communication configurationsaccording to embodiments of the present invention.

FIG. 10 is a perspective view depicting one embodiment of the presentinvention.

FIG. 11 is a two-dimensional graph of temperature readings fromhorizontal rows of sensors vs. distance along a lengthwise dimensionwithin a data center, according to an embodiment of the presentinvention.

FIG. 12 is a three-dimensional representation of the information shownin FIG. 11, according to an embodiment of the present invention.

FIG. 13 is a process flow diagram according to one embodiment of thepresent invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

It should be understood that these embodiments are only examples of themany advantageous uses of the innovative teachings herein. In general,statements made in the specification of the present application do notnecessarily limit any of the various claimed inventions. Moreover, somestatements may apply to some inventive features but not to others. Ingeneral, unless otherwise indicated, singular elements may be in theplural and vice versa with no loss of generality. In the drawing likenumerals refer to like parts through several views.

The present invention, according to an embodiment, overcomes problemswith the prior art by providing an efficient and easy-to-implementsystem and method for measuring the thermal distributions of a datacenter during operation.

Overview

In order to ascertain the thermal distributions of a data center underoperating conditions, a method and apparatus are disclosed to determinethe data center thermal properties as a function of location within thedata center. In an embodiment of the present invention, a plurality ofthermal sensors are mounted along a framework of a moveable cart. Eachsensor is placed at a defined distance from each other sensor on theframework.

A plurality of sensors is not necessarily located along a horizontalaxis. It may be, for example, oriented vertically or oriented along anyother combination of axes depending on the design of a particularapplication of a framework, as should be obvious to those of ordinaryskill in the art in view of the present discussion.

The cart is preferably taller than any of the racks in the data centerand has portions that extend above the racks. The sensors can capturetemperature measurements from near floor level to directly above theracks. The cart can be rolled through the data center while data loggingthe temperatures as a function of xyz coordinates. Thermal distributionsgenerated from the data can be used to make adjustments to the design orthe operation of the data center. Another useful application of thepresent invention makes it possible to redesign or adjust the coolingsystem in the data center as necessary. In other embodiments, the cartcan also capture wind speed, wind direction, relative and absolutehumidity, air pressure, and the like.

Data Center

An exemplary area suitable for use of the present invention is a datacenter. A data center is any room, area, volume, or part of a roomcontaining any kind of information technology equipment. An exemplarydata center 100 is illustrated in FIG. 1. The data center 100 isprovided with a cooling system, which is an elevated-floor coolingsystem. It should be noted, however, that there are alternateconfigurations for the cooling system, e.g. a non-raised floor system,where cool air is blown into the room via diffusers that get the airfrom air conditioning units via ducts and the hot air enters the a/cunits via the room. Although the remaining description discusses theelevated-floor cooling system, the present invention is useful for alltypes of cooling configurations.

As illustrated, a plurality of racks 102 are located on an elevatedfloor 104 so that there is room between the elevated floor 104 and asub-floor 106 to allow cold air 105 to circulate therein. Duringoperation, the racks 102 produce heat that is transferred to the ambientair to create hot air 109. The hot air 109 is sucked in by intakes 110in one or more chillers 108. The chillers 108 intake the air 109, heatedby the equipment of the data center 100, and transfer its heat to acooling fluid (which is typically an aqueous solution, or a refrigerant)circulating within the chiller 108. The chiller then outputs cold air105. The cold air is blown by the chillers 108 into a duct defined bythe raised floor 104 and the sub-floor 106. The cold air moves throughthe duct and exits through strategically placed vented openings 112 inthe surface of the elevated floor 104. The openings (i.e., perforatedtiles) 112 enable the cold air blown by the chillers 108 to exit theduct created by the raised floor 104 and sub-floor 106 and to enter anarea of the center 100 between rows of racks 102. As it can be seen fromFIG. 1 in a typical data center the aisles are divided into hot and coldaisles: The cold air 105 is moved into cold aisles through the openings112. The racks suck the cold air from the cold aisle into the racks 102to cool the various components. The hot air from the rack is then dumpedinto a hot aisle, which then is sucked into the AC 109.

The racks 102 are provided with vented areas 114 along their covers 115as well as fans or other circulation devices within the covers 115. Thefans draw ambient air (preferably from the cold aisles) through thevents 114, into the closed racks 102, and across the devices andcomponents within the racks 102. Vented areas on a side of the coveropposite the intake side of the cover allow the heated air to exit theracks 102, preferably to the cold aisle.

Obviously, the cooling effect of the air through a rack increases as theair temperature on the inlet side (cold aisle) of the rack decreases. Bystrategically placing the openings 112 in the raised floor 104 in closeproximity to the areas of greatest heat within the center 100, hotspots,or concentrations of heat, around one or more racks can be reduced.Ideally, the vented openings 112 in the raised floor 104 will bepositioned so that the center is balanced and all areas are atapproximately the same temperature. In order to balance the room,however, an accurate model of the temperature in each positionthroughout the room is desired. In addition, the arrangement and densityof the racks as well as the numbers of nodes within a rack can beoptimized to avoid hot spots in certain regions of the datacenter.

The Cart

In an embodiment of the present invention, illustrated in FIG. 2, a cart200 is defined by a framework of interconnected rods 202. Along the rods202 and at rod intersections 204 are mounted thermal sensors 206. Eachsensor 206 is at a defined distance from any other sensor 206. In oneembodiment, the present invention is provided with as many as 117sensors, although the number of sensors can be adjusted according to theapplication.

In the illustrated embodiment, the thermal sensors 206 are arranged suchthat they cover the corners of an imaginary unit cell, which extendsvertically and horizontally away from the sensor 206 with a distancethat is half the distance to the nearest adjacent sensor. In the figure,a unit cell 208, centered on a sensor 209, is defined by the distancebetween adjacent sensors 212, 214, and 216. If the dimensions of anexemplary first unit cell 208 is ⅔×⅔×1 feet, repeating the unit cell 208within the cart 200 allows the capture of a temperature reading with alateral (xy) resolution of ⅔ feet and 1 foot in the vertical (z)direction. It should be noted that these dimensions are only anillustrative example and other quantities of separation between thesensors can be used as well.

In one embodiment, the temperature sensors 206, or any other sensorsused, are thermally “isolated” from the cart 200. For example, thesensor can be separated from the cart 200 by a low thermal conductingmaterial, such as STYROPOR. Isolating the sensors from the cart ensuresthat the sensor readings reflect the ambient conditions in the datacenter and are not affected by the presence of the cart 200.

Measurement of physical quantities other than temperature may bedesirable. For instance, the measurement of wind speed, wind directionor relative and/or absolute humidity may be needed. A few exemplarysensors for capturing physical quantities are thermocouples, negativeand positive thermal resistive sensors, IR sensors, ceramic impedancemoisture sensors, thin film polymer capacitance sensors, anemometricsensors and acoustic sensors. Sensors for physical quantities such asthose mentioned are well known and the specifics of which will not bediscussed herein.

In a preferred embodiment, the rods 202 defining the cart 200 arerelatively thin so as to minimize the impact on wind flow when capturinga heat pattern. In one embodiment, the rod diameter is less than 1 inch,but can be other dimensions and may depend on material and desiredstrength. The rods 202 can be made of any rigid material that willstatically hold the sensors 206. However, in preferred embodiments, therod material is selected from a group of materials that have low thermalconductivity, such as plastic or composite material, that reduce apotential temperature influence on the sensors 206. In otherembodiments, the framework may be covered with a skin or other material.The term “framework” is not limited to only connected rods.

In one embodiment, the cart 200 is provided with a set of wheels 210that allow the cart 200 to easily be moved to any unoccupied positionwithin the center 100 so that measurements can be taken. Otherfriction-reducing devices can be used as well, such as castors, rollers,and the like. As shown in FIG. 3, the wheels 210 can be driven by one ormore motors 302 attached to one or more of the wheels 210 so thatoperator intervention is reduced or eliminated. The motors 302 can becontrolled by operator input, wired or wireless remote control, or wiredor wireless computer control. It is envisioned that the cart canautomatically move itself to every free tile in the data center andrecord temperatures without requiring operator input. While a cart 200is used in this example to move in the data center to take themeasurements, it should be obvious to those of ordinary skill in the artin view of the present discussion that other means for moving thesensors in the data center can be used without departing from theteachings of the present invention. For example, a wall mounted movingframework could be used to move sensors in the data center. As anotherexample, a ceiling mounted framework could be used to move sensors andto take the measurements in the data center. Additionally, anycombination of wall mounted and/or ceiling mounted framework orframeworks could be used to move sensors and to take the measurements inthe data center.

As illustrated in FIGS. 2 and 3, the cart 200, in this example, isshaped like a “T” in order to map the temperature distributions abovethe racks within the data center. The dimensions of the cart 200 aregoverned by the general data center layout and the rack dimensions. Forexample, as can be seen in FIG. 3, the cart 200 allows accuratemeasurement of a 9-foot high data center with a granularity of ⅔×⅔×1foot for racks 102 of up to 7.5 feet high and 4 feet deep. Thedimensions given are exemplary only. Other dimensions can be used andare within the true spirit and scope of the present invention.

FIGS. 4-7 show how the cart can be used to measure the three-dimensionaldistribution of a physical quantity, such as temperature, throughout thedata center 100. In FIG. 4, a cart 400 rests on a floor 402 and ispositioned adjacent to a rack 404 so that the right side 406 of the “T”section of the cart 400 is extended above the rack 404. In thisconfiguration, the sensors along the main tower 408 of the cart 400, aswell as the sensors on the right side 406 of the “T,” are utilized tocapture the physical quantity being measured. The sensors on the leftside 410 of the “T” are not really needed for this measurement.

In FIG. 5, the cart 400 is moved to the opposite side of the rack 404 ofFIG. 4. In this configuration, the left side 410 of the “T” is extendingdirectly above the rack 404. Ideally, the “T” portion of the cart 400 isat least half the width of the rack 404, so that all regions above therack 404 can be measured. In the configuration of FIG. 5, the sensorslocated in the left side 410 of the “T” section are utilized, while thesensors in the right side 406 of the “T” section are typically not.Sensors within the main tower section 408 are utilized.

FIG. 6 shows the cart 400 located between two racks 404 & 600. Asillustrated in FIG. 6, the “T” section of the cart 200 does not extendabove either rack 400 or 600. In this configuration, the area beingcaptured is from the floor 602 to the ceiling 604. Therefore, thesensors in the center tower section of the cart are able to capture allnecessary measurements within this physical space. The “T” sections arenot needed.

In FIG. 7, the cart 200 is moved one tile to the right so that the rightside 406 of the “T” section 406 extends above the rack 600. Thisconfiguration is the same as described above, with reference to FIG. 4.The sensors along the central main tower 408 of the cart 400, as well asthe sensors on the right side 406 of the “T,” are utilized to capturethe physical quantity being measured. The sensors on the left side 410of the “T” are not needed and are not used for this measurement as theywould be redundant with the measurement taken in FIG. 6.

One solution to reducing hotspots within the data center is to place avented tile on a portion of the raised floor near the hotspot. In apreferred embodiment, the foot print of the cart is equal to or lessthan the dimensions of a tile on the raised floor 104. This granularityallows for better analysis and modeling of the effects of exchangingtiles.

In another embodiment of the present invention, the sensors are movablein relation to the framework of the cart. In this embodiment, thesensors record a measurement at a first location, move to a secondlocation where they record a second measurement, move to a thirdlocation, and so on. This embodiment reduces the number of sensorsneeded and, therefore, reduces costs and failures.

In an additional embodiment of the present invention, the framework isextendable so that a single cart can accommodate a number of areashaving different dimensions. The framework can be telescoping, foldable,attachable to additional framework pieces, or other similar methods ofextension.

Computer System

As stated above, the present invention can be moved on wheels to everyunoccupied tile in the data center where measurements are taken. Inorder to create a spatially accurate thermal characterization of thedata center, specific sensors must be individually addressed, read, andlogged at each location within the center. Additionally, lateralrotation of the cart should be detected, tracked, and logged. Referringback to FIG. 3, a computer system 304 is located on the cart 300.

FIG. 8 is a block diagram of a computer system with which an embodimentof the present invention can be implemented. A computer system mayinclude, inter alia, one or more computers or computer cards and atleast a computer readable medium, allowing a computer system, to readdata, instructions, messages or message packets, and other computerreadable information from the computer readable medium. The computersystem includes one or more processors, such as processor 804. Theprocessor 804 can be wired or wirelessly connected to a communicationinfrastructure 802, e.g., a network. After reading this description, itwill become apparent to a person of ordinary skill in the relevantart(s) how to implement the invention in other computer systems and/orcomputer architectures.

The computer system may include a display interface 808 that forwardsdata from the communication infrastructure 802 for display on thedisplay unit 810. The computer system also includes a main memory 806,preferably random access memory (RAM), and may also include a secondarymemory 812. The secondary memory 812 may include, for example, a harddisk drive 814 and/or a removable storage drive 816, representing afloppy disk drive, a magnetic tape drive, an optical disk drive, flashmemory etc. The removable storage drive 816 may read from and/or writeto a removable storage unit 818 in a manner well known to those havingordinary skill in the art. Removable storage unit 818, represents afloppy disk, magnetic tape, optical disk, flash memory, etc. which isread by and written to by removable storage drive 816. As will beappreciated, the removable storage unit 818 includes a computer usablestorage medium having stored therein computer software and/or data.

In alternative embodiments, the secondary memory 812 may include othersimilar means for allowing computer programs or other instructions to beloaded into the computer system. Such means may include, for example, aremovable storage unit 822 and an interface 820. Examples of such mayinclude a program cartridge and cartridge interface (such as that foundin video game devices), a removable memory chip (such as an EPROM, orPROM) and associated socket, and other removable storage units 822 andinterfaces 820 which allow software and data to be transferred from theremovable storage unit 822 to the computer system.

The computer system may also include a communications interface 824.Communications interface 824 allows software and data to be transferredbetween the computer system and external devices, such as temperaturesensors. Examples of communications interface 824 may include a modem, anetwork interface (such as an Ethernet card), a communications port, aPCMCIA slot and card, etc. Software and data transferred viacommunications interface 824 are in the form of signals which may be,for example, electronic, electromagnetic, optical, or other signalscapable of being received by communications interface 824. These signalsare provided to communications interface 824 via a communications path(i.e., channel) 826. This channel 826 carries signals and may beimplemented using wire or cable, fiber optics, a phone line, a cellularphone link, an RF link, and/or other communications channels.

In this document, the terms “computer program medium,” “computer usablemedium,” and “computer readable medium” are used to generally refer tomedia such as main memory 806 and secondary memory 812, removablestorage drive 816, a hard disk installed in hard disk drive 814, andsignals. These computer program products are means for providingsoftware to the computer system. The computer readable medium allows thecomputer system to read data, instructions, messages or message packets,and other computer readable information from the computer readablemedium. The computer readable medium, for example, may includenon-volatile memory, such as Floppy, ROM, Flash memory, Disk drivememory, CD-ROM, and other permanent storage. It is useful, for example,for transporting information, such as data and computer instructions,between computer systems. Furthermore, the computer readable medium maycomprise computer readable information in a transitory state medium suchas a network link and/or a network interface, including a wired networkor a wireless network that allow a computer to read such computerreadable information.

In one embodiment, each sensor is individually addressable. Each sensoris provided with a means for communication which includes wired orwireless communication. In a wired embodiment, each sensor connects tothe communications interface 824 that allows the sensor measurements tobe recorded by the computer system located on the framework or off. Inother embodiments, as shown in FIG. 9, the sensor readings 902 aretransmitted wirelessly to a remote computer system 306 byelectromagnetic radiation, optical, sound, or other similar means ofwireless communication. In still other embodiments, the data istransmitted, either wired or wirelessly, to a network 308, where thedata is accessible to multiple computer systems 310 a-310 n forevaluation, storage, and other uses.

In one embodiment of the present invention, multiplex electronics areutilized to capture and record the sensor measurements. In thisembodiment, all the sensor readouts are multiplexed using the multiplexelectronics and then read by one analog to digital (A/D) card into thecomputer for storage, analysis, or manipulation.

In other cases, instead of using a complete computer system to store thedata, it can be deposited onto a memory device and then off-loadedlater. In short, any method of transmitting sensor readings to aninformation processing unit or memory device is contemplated and iswithin the spirit and scope of the present invention.

Software

Computer programs (also called computer control logic) are stored inmain memory 406 and/or secondary memory 412. Computer programs may alsobe received via communications interface 424. Such computer programs,when executed, enable the computer system to perform the features of thepresent invention as discussed herein. In particular, the computerprograms, when executed, enable the processor 404 to perform thefeatures of the computer system. Accordingly, such computer programsrepresent controllers of the computer system.

Once the data is recorded and stored, software is used to read the datafiles and to manipulate or utilize the data as needed. An example offortran-type computer code used in the present invention is given underthe section entitled “Example of a Fortran-Type Computer Code”. Itshould be noted that the computer code shown is for exemplary purposesonly and the invention is not so limited. Additionally, the programshown in the section entitled “Example of a Fortran-Type Computer Code”,or other equivalent methods, can be used to characterize quantitiesother than temperature, such as humidity, wind speed, pressure, and thelike.

In this example, temperature data is read from one or more sensors ateach free tile in the data center. The readings taken at each tile arethen stored separately. Each data file consists of a single column of nindividual temperature values received from the same number n of sensorsread at a given tile location. An exemplary order of temperature sensorson the cart is shown in FIG. 2. In addition, there is a netlist file,which has stored the filename of each data file (from eachalready-measured tile location), the corresponding tile coordinates (x,y), the orientation (E, W, N, S) and which sides of the arms of the T isused (right, left, both, or nothing). The netlist file also includes anumber indicating in which directory the data file has been stored.

The program first reads all the individual files, then it puts the datainto three different three-dimensional arrays: the center block (cb)with 3×3×9 data points using the center of the “T” (sensors 1 to 81),the left block (lb) with 3×3×2 data points using the left arm of the “T”(sensors 82 to 99) and finally the right block (rb), with 3×3×2 datapoints using the right arm of the “T” (sensors 100 to 117).

Using the orientation information (such as lateral movement) of thecart, the blocks are rotated laterally so that the same coordinatesystem is used throughout the data center. For instance, if theexemplary cart 200, shown in FIG. 2, is rotated 90 degreescounterclockwise, as shown in FIG. 10, the sensors 1-117 exchangeplaces. Specifically, looking at sensors 1-9 on the lowest row, sensor 5remains in the same position, while all other sensors rotatecounterclockwise around sensor 5. As a result, after the lateralrotation, sensor 1 is where sensor 7 used to be, sensor 2 is wheresensor 4 used to be, sensor 3 is where sensor 1 used to be, and so on.These new positions must be tracked so that when the sensor readings arelater compiled, the positions can be factored into the calculations.Therefore, when the cart 200 is rotated, the data blocks (cb, lb, rb)have to be mathematically laterally rotated to correspond to the neworientation of the cart. Using the tile information (x and y coordinatesof the tiles from the netlist file) the data blocks are stored into theright position of a global array (gtemp) which contains all the data inall three dimensions throughout the data center in the same coordinatesystem.

Rotation and position of the cart can be sensed by a position sensor. Afew examples of position sensors are a global positioning system (GPS),a differential global positioning system (DGPS), which utilizes a radiolink between a stationary GPS and a mobile GPS, a compass, one or moresensors coupled to the wheels of the cart, or other similar devices,which can all be contained within the computer 304 or other locations onthe framework of the cart. The position and orientation data may be readand stored while data logging the sensors at a given location within thedata center.

In some applications it is desired to control the motors 302 to move thecart by using the positioning and orientation data. A computer system,as described below, is able to coordinate this feedback. In addition thecart 200 could be equipped with one or more proximity sensors 218 (LEDs,bump sensors etc.) to help to control the motion. After each measurementis taken, the cart uses the position data, orientation data, and/ornavigation sensors to move to a new open tile on the floor. The processrepeats until every tile location has been measured. The additionalsensors reduce or remove the need for operator interaction.

Once the data is collected, it can be rapidly organized into a formatthat is easy to view and manipulate. For instance, the data can be usedto perform spatial data slices, generate contour plots & histograms,images, statistical output, generate single or multi-dimensional graphs,and others. It is one goal of the present invention to graphically andaccurately represent the temperatures within the room as a function ofthe xy coordinates of the measurements.

In some applications the data is then transferred to a thermal expert,which can be a set of guidelines, a system that is able to evaluate theinformation and make decisions based on the data, or an actual person.The data can be transferred to the expert via Internet, wiredcommunication, wireless communication, by physical transport of data ondisk, or any other method of transferring data as is obvious to those ofordinary skill in the art in view of the present discussion.

After being evaluated by the thermal expert, recommendations arecommunicated to a location implementer that will modify the data center,if necessary, accordingly. Some of the adjustment options are, forexample, to move, add, or remove perforated tiles, to move or removeracks within the data center, to move, add, or remove ducting of thecold air, and to move, add, or remove rear cover heat exchangers. Theadjustment options are not limited to the previous choices, which areprovided for illustration purposes and not for any limitation of themany different adjustment options, and can also include other optionsthat will alter the environment as needed. By graphically representingthe data center temperatures, or other aspects, cooling strategies,options, and locations can be intelligently selected.

Application of the Invention

As stated above, within an exemplary data center are a plurality ofracks that hold heat-producing computer equipment. One or more coolingsystems are used to control and remove heat produced by the equipment.Examples of cooling systems are overhead air ducting, side air ducting,under-floor air ducting that exit through vented tiles in the floorsurface between the racks, internal refrigeration systems, and the like.Temperature information can be obtained by taking a single measurementor multiple measurements within the center. Additionally, more accuratetemperature profiles can be obtained by taking consecutive measurementsat locations that are in substantially close proximity to eachpreviously measured location. In one embodiment, for ease of uniformity,the inventive cart is placed on each tile within a range of tiles thatare located between two rows of racks. As an example, the range of tilesbetween the two rows of racks is 84 tiles in length and 4 tiles inwidth. Each tile is 2′×2′ in size. At each tile location a temperaturemeasurement is taken and data logged as a function of the xyzcoordinates within the room. The inventive cart can be provided with anynumber of sensors and can be any shape or dimension. In this example,the cart is of the dimensions shown in FIG. 3.

Once data, whether temperature or otherwise, is detected by the sensorson the cart, the data is communicated either by wire or wirelessly to acomputer or memory device where it is labeled and stored. By assigningcolors or patterns to a range of temperatures, a graphically meaningfulrepresentation of the readings can be displayed. For instance, agraphical representation of the temperatures reported from the lowestrow of sensors (1-9 as shown in FIG. 2) of a cart placed on each tile inthe range of tiles can be displayed. In one example, the temperaturesrange from 18.4 degrees C. to 47.7 degrees C. Each successive row ofsensors reports temperatures or other measurements at each incrementallyincreasing height until a maximum height of 8.5 feet is reached. Ofcourse exact dimensions are mentioned for exemplary purposes only andthe invention is not so limited.

FIG. 11 shows a two-dimensional graph of the temperature readings ofeach horizontal row of sensors vs. distance along the lengthwisedimension of the range of tiles 908, according to the present example.Each line type is different and represents an individual horizontal rowof sensors within the cart. The chart of FIG. 11 shows a single slice ofthe temperatures in the data center. In other words, the chart shows onepass of the cart along the 84 tiles. This type of graph can be repeatedfor each of the four rows of tiles, or any other resolution, between thetwo rows of racks 910 and 912. Additionally, the upper portion of thecart is a “T” shape that extends above the racks 908. Therefore, a chartcan be also be created to show temperatures above the racks.

Referring now to FIG. 12, the data represented in the two-dimensionalgraph of FIG. 11 is shown in a three-dimensional representation 1200. Aset of temperatures ranging from 18.4 degrees C. to 47.7 degrees C. isrepresented in the graph with each temperature range being representedby different shadings. In the three dimensional graph of FIG. 12, it cannow easily be seen that the hot-spot areas within the center areadjacent the racks 902. Specifically, the hottest areas are not near thefloor, where perforated tiles are allowing cool air to exit the ductedfloor, but at a height near the top or above the racks. The threedimensional graph of FIG. 12 allows one to immediately see problem areasand plan cooling strategies around these areas. It is noted, however,that the graph of FIG. 12 shows the data captured with only one passdown the data center floor with the inventive cart. As will now be seen,the present invention advantageously allows for even more accurateservicing of spatial temperature characteristics within a data center.

As previously described, the “T” shaped cart can be moved to everyunoccupied tile in the data center to capture readings. The cartdimensions are selected so that the “T” portion of the cart is able toextend over the racks and capture temperature or other readings abovethe racks as well. The data from the sensors, as well as xy coordinatesand orientation data of the cart, is communicated to a computer thatcompiles the data and is able to output the data in various formats,such as a format that can be displayed graphically.

Process Flow

FIG. 13 shows a process flow chart of one embodiment of the presentmeasurement system. The flow begins at step 1300 and moves directly tostep 1302, where the cart is placed on a first tile in an area to bemeasured. Sensors on the cart collect measurements in step 1304. Thecollected measurements, along with xy coordinates, cart location andorientation information, are then transferred to a measurement storagedevice in step 1306. In decision step 1308, a check is performed to seeif any other tiles need to be measured. If the answer is “yes,” the flowmoves to step 1310 where the cart is moved to a new tile location. Theflow then moves back to step 1304 where sensors collect new measurementsabove the new tile location. The flow continues as described above untilthe decision step 3308 is reached again.

If the answer to decision step 1308 is “no,” flow moves to step 1312where the data from all the sensors in all the tile locations istransmitted to a thermal expert. The thermal expert, in step 1314,evaluates the data. After the data has been evaluated, the thermalexpert makes a recommendation for a cooling solution in step 1316. Thecooling solution is implemented in step 1318. Decision step 1320 askswhether the room needs to be reevaluated to determine the effectivenessof the cooling solution implemented. If the answer is “yes,” the flowmoves back up to step 1302 where the cart is placed back on the firsttile. In practice, the tile order in unimportant and the data can betaken in any order. If the answer to decision step 1320 is “no,” theflow moves to step 1322 and stops.

It should be noted that with each iteration of implementing the thermalexpert's recommendations, the region of interest shrinks smaller andsmaller. It is contemplated that each iteration of measurements will betaken around a smaller region than the previous measurement.

CONCLUSION

Although specific embodiments of the invention have been disclosed,those having ordinary skill in the art will understand that changes canbe made to the specific embodiments without departing from the spiritand scope of the invention. The scope of the invention is not to berestricted, therefore, to the specific embodiments. Furthermore, it isintended that the appended claims cover any and all such applications,modifications, and embodiments within the scope of the presentinvention.

“Example of a Fortran-Type Computer Code” nx=31 ; *** total tilex-coordinates ny=29 ; *** total tile y-cooridnates backtemp=20.0 ; ***background temp dnu_files=579 ; *** number of files in the netlistgtemp=fltarr(nx*3,ny*3,9) gtemp(*,*,*)=backtemp cb=fltarr(3,3,9)lb=fltarr(3,3,2) rb=fltarr(3,3,2) ; *** read netlist ; *** co(0,*)filename ; *** co(1,*) tile x-coordinate ; *** co(2,*) tile y-coordinate; *** co(3,*) orientation of the cart (E,W,N,S) ; *** co(4,*) is whichsides are used (R,L) ; *** co(5,*) directory of the files ; for 0 nosides are used ; for 1 left side is used ; for 2 right side is usedco=strarr(7,nu_files)status=dc_read_free(‘c:\...\netlist.txt’,co,/Column,Delim=‘\011’) ; ***assume that netlist is saved from origin with tab as delim ; *** readall the temperature files and put them into one block for i=0,nu_files−1do begin  if co(5,i) EQ ‘1’ thenname=strcompress(‘c:\...\POK_DataCenter1_0517\’+co(0,i)+‘.txt’,/remove_all) if co(5,i) EQ ‘2’ thenname=strcompress(‘c:\...\POK_DataCenter_0518\’+co(0,i)+‘.txt’,/remove_all) if co(5,i) EQ ‘3’ thenname=strcompress(‘c:\...\POK_DataCenter_0519\’+co(0,i)+‘.txt’,/remove_all) status=dc_read_free(name,d)  cb(*,*,*)=0.0  lb(*,*,*)=0.0 rb(*,*,*)=0.0 ; *** for the center block  q=0  for iz=0,8 do begin  forix=0,2 do begin   for iy=0,2 do begin   cb(ix,iy,iz)=d(q)   q=q+1   end end  end ; *** for the left block  if co(4,i) EQ ‘L’ or co(4,i) EQ‘R,L’ or co(4,i) EQ ‘L,R’ then begin  q=81  for iz=0,1 do begin  forix=0,2 do begin   for iy=0,2 do begin   lb(ix,iy,iz)=d(q)   q=q+1   end end  end  end ; *** for the right block  if co(4,i) EQ ‘R’ or co(4,i)EQ ‘R,L’ or co(4,i) EQ ‘L,R’then begin  q=99  for iz=0,1 do begin  forix=0,2 do begin   for iy=0,2 do begin   rb(ix,iy,iz)=d(q)   q=q+1   end end  end  end  if co(4,i) EQ ‘L’ then print, lb(*,*,*)  tx=fix(co(1,i)) ty=fix(co(2,i))  if co(3,i) EQ ‘W’ then ori=0  if co(3,i) EQ ‘S’ thenori=1  if co(3,i) EQ ‘E’ then ori=2  if co(4,i) EQ ‘N’ then ori=3 gtemp(tx*3:tx*3+2,ty*3:ty*3+2,*)=rotate(cb(*,*,*),ori) ; *** left andright for west (0)  if co(4,i) EQ ‘L’ and ori EQ 0 then begin gtemp((tx−1)*3:(tx−1)*3+2,ty*3:ty*3+2,7:8)=rotate(lb(*,*,*),ori)  end if co(4,i) EQ ‘R’ and ori EQ 0 then begin gtemp((tx+1)*3:(tx+1)*3+2,ty*3:ty*3+2,7:8)=rotate(rb(*,*,*),ori)  end if (co(4,i) EQ ‘R,L’ and ori EQ 0) or (co(4,i) EQ ‘L,R’ and ori EQ 0)then begin gtemp((tx−1)*3:(tx−1)*3+2,ty*3:ty*3+2,7:8)=rotate(lb(*,*,*),ori) gtemp((tx+1)*3:(tx+1)*3+2,ty*3:ty*3+2,7:8)=rotate(rb(*,*,*),ori)  end ;*** left and right for south (1)  if co(4,i) EQ ‘L’ and ori EQ 1 thenbegin  gtemp(tx*3:tx*3+2,(ty−1)*3:(ty−1)*3+2,7:8)=rotate(lb(*,*,*),ori) end  if co(4,i) EQ ‘R’ and ori EQ 1 then begin gtemp(tx*3:tx*3+2,(ty+1)*3:(ty+1)*3+2,7:8)=rotate(rb(*,*,*),ori)  end if (co(4,i) EQ ‘R,L’ and ori EQ 1) or (co(4,i) EQ ‘L,R’ and ori EQ 1)then begin gtemp(tx*3:tx*3+2,(ty−1)*3:(ty−1)*3+2,7:8)=rotate(lb(*,*,*),ori) gtemp(tx*3:tx*3+2,(ty+1)*3:(ty+1)*3+2,7:8)=rotate(rb(*,*,*),ori)  end ;*** left and right for east (2)  if co(4,i) EQ ‘L’ and ori EQ 2 thenbegin  gtemp((tx+1)*3:(tx+1)*3+2,ty*3:ty*3+2,7:8)=rotate(lb(*,*,*),ori) end  if co(4,i) EQ ‘R’ and ori EQ 2 then begin gtemp((tx−1)*3:(tx−1)*3+2,ty*3:ty*3+2,7:8)=rotate(rb(*,*,*),ori)  end if (co(4,i) EQ ‘R,L’ and ori EQ 2) or (co(4,i) EQ ‘L,R’ and ori EQ 2)then begin gtemp((tx+1)*3:(tx+1)*3+2,ty*3:ty*3+2,7:8)=rotate(lb(*,*,*),ori) gtemp((tx−1)*3:(tx−1)*3+2,ty*3:ty*3+2,7:8)=rotate(rb(*,*,*),ori)  end ;*** left and right for north (3)  if co(4,i) LT ‘L’ and ori EQ 3 thenbegin  gtemp(tx*3:tx*3+2,(ty+1)*3:(ty+1)*3+2,7:8)=rotate(lb(*,*,*),ori) end  if co(4,i) GT ‘R’ and ori EQ 3 then begin gtemp(tx*3:tx*3+2,(ty−1)*3:(ty−1)*3+2,7:8)=rotate(rb(*,*,*),ori)  end if (co(4,i) EQ ‘R,L’ and ori EQ 3) or (co(4,i) EQ ‘L,R’ and ori EQ 3)then begin gtemp(tx*3:tx*3+2,(ty+1)*3:(ty+1)*3+2,7:8)=rotate(lb(*,*,*),ori) gtemp(tx*3:tx*3+2,(ty−1)*3:(ty−1)*3+2,7:8)=rotate(rb(*,*,*),ori)  endend print,max(gtemp(*,*,*)), min(gtemp(*,*,*)) ; *** xy display all 9layers in the data center with scaling for i=0,8 do begin Window,i,XSize=12*nx, YSize=12*ny  TV,bytscl(smooth(rebin(gtemp(*,*,i),12*nx,12*ny),4),Min=min(gtemp(*,*,*)),Max=max(gtemp(*,*,*))) end ; *** xy display all 7 layers in the data center withscaling test=fltarr(252,6) for i=0,6 do begin Window,i,XSize=504,YSize=12 test=gtemp(*,3:8,i) TV,bytscl(smooth(rebin(test,504,12),6),Min=min(gtemp(*,*,*)),Max=max(gtemp(*,*,*))) end ; *** display line scans through the data center ; *** x-directionv1x=fltarr(252) v2x=fltarr(252) voutx=fltarr(9,252) for i=0,8 do begin v1x(*)=gtemp(*,5,i)  v2x(*)=gtemp(*,6,i) voutx(i,*)=smooth((v1x(*)+v2x(*))/2,3) endstatus=dc_write_free(‘c:\...\linex.txt’,voutx(*,*),/column) ; *** xzdisplay middle layers in the data center with scaling h=fltarr(252,9)Window,0, XSize=504,YSize=18 h(*,*)=gtemp(*,5,*) info,hTVSCL,smooth(rebin(h,504,18),6) End

1. An apparatus for measuring physical characteristics, the apparatuscomprising: a framework having a plurality of sensors, each sensor beingat a different location on the framework such that the plurality ofsensors defines one three-dimensional unit cell, and each sensor formeasuring a physical characteristic of an environment within a datacenter such that the plurality of sensors is for contemporaneouslymeasuring the physical characteristic in the one three-dimensional unitcell within the data center, and wherein at least one sensor of theplurality of sensors is thermally isolated from the framework; means formoving the framework through the data center at defined incrementscorresponding to one or more dimensions of the one three-dimensionalunit cell such that the plurality of sensors is located at increments ofthe three-dimensional unit cell to measure the physical characteristicof the environment; and means for communicating at least one of: themeasured physical characteristic from at least one of the plurality ofsensors; location information within the data center of at least one ofthe framework and a sensor on the framework, based on movement of theframework within the data center, wherein the location informationcorresponds to the set of defined increments; and orientationinformation of the framework wherein the orientation informationcorresponds to the set of defined increments, to a data storage device.2. The apparatus of claim 1, wherein the framework is shaped anddimensioned so that at least one of the plurality of sensors is able tomeasure a physical characteristic at at least one of directly above arack in the data center and on a side of a rack in the data center. 3.The apparatus of claim 1, wherein the plurality of sensors comprises atleast one sensor for measuring at least one of temperature, relativehumidity, absolute humidity, barometric pressure, wind speed, and winddirection.
 4. The apparatus of claim 1, further comprising: at least onefriction-reducing device coupled to the framework allowing the frameworkto be moved to locations within the data center.
 5. The apparatus ofclaim 4, further comprising: at least one motor coupled to the at leastone friction-reducing device, the at least one motor for moving theframework within the data center.
 6. The apparatus of claim 1, furthercomprising: at least one of a location sensing device, an orientationsensing device, and a proximity sensing device coupled to the framework.7. The apparatus of claim 6, wherein an output from one of the locationsensing device, the orientation sensing device, and the proximitysensing device is used to control at least one motor to move theframework through the data center.
 8. The apparatus of claim 6, whereinthe location sensing device comprises a global positioning system andthe orientation sensing device comprises a compass.
 9. The apparatus ofclaim 1, wherein the means for communicating comprises a wirelesstransmitter.
 10. The apparatus of claim 1, wherein the data storagedevice comprises at least one of a computer readable medium, a computermemory, a hard disk drive, a flash memory, a remote computer system, anda remote network.